Graphical Multi%2Dway Models.

Variational Bayesian Mixture of Robust CCA Models.

Inferring object relevance from gaze in dynamic scenes.

An information retrieval perspective on visualization of gene expression data with ontological annotation.

Hierarchical Generative Biclustering for MicroRNA Expression Analysis.

Three Paths to Relevance.

Global modeling of transcriptional responses in interaction networks.

Multivariate multi%2Dway analysis of multi%2Dsource data.

Relevant subtask learning by constrained mixture models.
models
Information Retrieval Perspective to Nonlinear Dimensionality Reduction for Data Visualization.
Nonlinear dimensionality reduction methods are often used to visualize high-dimensional data, although the existing methods have been designed for other related tasks such as manifold learning. It has been difficult to assess the quality of visualizations since the task has not been well-defined. We give a rigorous definition for a specific visualization task, resulting in quantifiable goodness measures and new visualization methods. The task is information retrieval given the visualization: to find similar data based on the similarities shown on the display. The fundamental tradeoff between precision and recall of information retrieval can then be quantified in visualizations as well. The user needs to give the relative cost of missing similar points vs. retrieving dissimilar points, after which the total cost can be measured. We then introduce a new method NeRV (neighbor retrieval visualizer) which produces an optimal visualization by minimizing the cost. We further derive a variant for supervised visualization; class information is taken rigorously into account when computing the similarity relationships. We show empirically that the unsupervised version outperforms existing unsupervised dimensionality reduction methods in the visualization task, and the supervised version outperforms existing supervised methods.
Infinite factorization of multiple non%2Dparametric views.
be inserted by the editor)
Automatic Choice of Control Measurements.

Using dependencies to pair samples for multi%2Dview learning.
Informaatio- ja luonnontieteiden tiedekunta
Two%2DWay Analysis of High%2DDimensional Collinear Data.

Using dependencies to pair samples for multi%2Dview learning.
Informaatio- ja luonnontieteiden tiedekunta
Supervised nonlinear dimensionality reduction by Neighbor Retrieval.

GaZIR gaze%2Dbased zooming interface for image retrieval.

Two%2DWay Grouping by One%2DWay Topic Models.
Abstract. We tackle the problem of new users or documents in collaborative filtering. Generalization over users by grouping them into user groups is beneficial when a rating is to be predicted for a relatively new document having only few observed ratings. The same applies for documents in the case of new users. We have shown earlier that if there are both new users and new documents, two-way generalization becomes necessary, and introduced a probabilistic Two-Way Model for the task. The task of finding a two-way grouping is a non-trivial combinatorial problem, which makes it computationally difficult. We suggest approximating the Two-Way Model with two URP models; one that groups users and one that groups documents. Their two predictions are combined using a product of experts model. This combination of two one-way models achieves even better prediction performance than the original Two-Way Model. 1
Bayesian Solutions to the Label Switching Problem.
Abstract. The label switching problem, the unidentifiability of the permutation of clusters or more generally latent variables, makes interpretation of results computed with MCMC sampling difficult. We introduce a fully Bayesian treatment of the permutations which performs better than alternatives. The method can even be used to compute summaries of the posterior samples for nonparametric Bayesian methods, for which no good solutions exist so far. Although being approximative in that case, the results are very promising. Thesummariesareintuitivelyappealing: A summarized cluster is defined as a set of points for which the likelihood of being in the same cluster is maximized. 1
Probabilistic retrieval and visualization of biologically relevant microarray experiments.

Probabilistic retrieval and visualization of biologically relevant microarray experiments.

Visualizations for assessing convergence and mixing of Markov chain Monte Carlo simulations.
NOTICE: this is the author’s version of a work that was accepted for publication
Latent grouping models for user preference prediction.
We introduce a novel latent grouping model  for predicting the relevance of a new document  to a user. The model assumes a latent  group structure for both users and documents.
Two%2Dway analysis of high%2Ddimensional collinear data.

Latent grouping models for user preference prediction.
We introduce a novel latent grouping model  for predicting the relevance of a new document  to a user. The model assumes a latent  group structure for both users and documents.


Learning to learn implicit queries from gaze patterns.
In the absence of explicit queries, an alternative is to try to infer users ’ interests from implicit feedback signals, such as clickstreams or eye tracking. The interests, formulated as an implicit query, can then be used in further searches. We formulate this task as a probabilistic model, which can be interpreted as a kind of transfer or meta-learning. The probabilistic model is demonstrated to outperform an earlier kernel-based method in a small-scale information retrieval task. 1.
Simple integrative preprocessing preserves what is shared in data sources.
Simple integrative preprocessing preserves what is shared in data sources
An Analysis of Generalization Error in Relevant Subtask Learning.

Can relevance of images be inferred from eye movements%3F

Simple integrative preprocessing preserves what is shared in data sources.
Simple integrative preprocessing preserves what is shared in data sources
Learning from Relevant Tasks Only.
Abstract. We introduce a problem called relevant subtask learning, a variant of multi-task learning. The goal is to build a classifier for a taskof-interest having too little data. We additionally have data for other tasks but only some are relevant, meaning that they contain samples classified in the same way as in the task-of-interest. The problem is how to utilize this “background data ” to improve the classifier in the task-ofinterest. We show how to solve the problem for logistic regression classifiers, and demonstrate that the solution works better than a comparable multi-task learning model. The key is to assume that data of all tasks are mixtures of relevant and irrelevant samples, and model the irrelevant part with a sufficiently flexible model such that it does not distort the model of relevant data. Key words: multi-task learning, partially relevant data, relevant subtask learning, transfer learning 1
Functional elements and networks in fMRI.
Abstract. We propose a two-step approach for the analysis of functional magnetic resonance images, in the context of natural stimuli. In the first step, elements of functional brain activity emerge, based on spatial independence assumptions. The second step exploits temporal covariation between the elements and given features of the natural stimuli to identify functional networks. The networks can have complex activation patterns related to common task goals. 1
Local dependent components.
We introduce a mixture of probabilistic canonical correlation analyzers model for analyzing local correlations, or more generally mutual statistical dependencies, in cooccurring data pairs. The model extends the traditional canonical correlation analysis and its probabilistic interpretation in three main ways. First, a full Bayesian treatment enables analysis of small samples (large p, small n, a crucial problem in bioinformatics, for instance), and rigorous estimation of the degree of dependency and independency. Secondly, the mixture formulation generalizes the method from global linearity to the more reasonable assumption of different kinds of dependencies for different kinds of data. As a third novel extension the method decomposes the variation in the data into shared and data set-specific components. 1.
Inferring Vertex Properties from Topology in Large Networks.
large networks
Methods for estimating human endogenous retrovirus activities from EST databases.

Probabilistic modeling and machine learning in structural and systems biology.

Neural networks and machine learning in bioinformatics %2D theory and applications.

Comparison of visualization methods for an atlas of gene expression data sets.
ISBN 978-951-22-8751-2 (printed version) ISBN 978-951-22-8752-9 (electronic version)
Neural networks and machine learning in bioinformatics %2D theory and applications.

Visualizing gene interaction graphs with local multidimensional scaling.
Reprinted with permission.
Local multidimensional scaling.
Local multidimensional scaling 2 Local multidimensional scaling In a visualization task, every nonlinear projection method needs to make a compromise between trustworthiness and continuity. In a trust-worthy projection the visualized proximities hold in the original data as well, whereas a continuous projection visualizes all proximities of the original data. We show experimentally that one of the multidi-mensional scaling methods, curvilinear components analysis, is good at maximizing trustworthiness. We then extend it to focus on local proximities both in the input and output space, and to explicitly make a user-tunable parameterized compromise between trustworthiness and continuity. The new method compares favorably to alternative nonlin-ear projection methods.
On Discriminative Joint Density Modeling.
We study discriminative joint density models, that is, generative  models for the joint density p(c, x) learned by maximizing a discriminative  cost function, the conditional likelihood. We use the framework  to derive generative models for generalized linear models, including  logistic regression, linear discriminant analysis, and discriminative mixture  of unigrams. The benefits of deriving the discriminative models from  joint density models are that it is easy to extend the models and interpret  the results, and missing data can be treated using justified standard  methods.
Implicit Relevance Feedback from Eye Movements.
We explore the use of eye movements as a source of implicit  relevance feedback information. We construct a controlled information  retrieval experiment where the relevance of each text is known, and test  usefulness of implicit relevance feedback with it. If perceived relevance  of a text can be predicted from eye movements, eye movement signal  must contain information on the relevance. The result is that relevance  can be predicted to a considerable extent with discriminative hidden  Markov models, and clearly better than randomly already with simple  linear models of time-averaged data.
Expectation maximization algorithms for conditional likelihoods.
We introduce an expectation maximizationtype (EM) algorithm for maximum likelihood optimization of conditional densities. It is applicable to hidden variable models where the distributions are from the exponential family. The algorithm can alternatively be viewed as automatic step size selection for gradient ascent, where the amount of computation is traded off to guarantees that each step increases the likelihood. The tradeoff makes the algorithm computationally more feasible than the earlier conditional EM. The method gives a theoretical basis for extended Baum Welch algorithms used in discriminative hidden Markov models in speech recognition, and compares favourably with the current best method in the experiments.
Combining eye movements and collaborative filtering for proactive information retrieval.
We study a new task, proactive information retrieval by combining implicit relevance feedback and collaborative filtering. We have constructed a controlled experimental setting, a prototype application, in which the users try to find interesting scientific articles by browsing their titles. Implicit feedback is inferred from eye movement signals, with discriminative hidden Markov models estimated from existing data in which explicit relevance feedback is available. Collaborative filtering is carried out using the User Rating Profile model, a state-of-the-art probabilistic latent variable model, computed using Markov Chain Monte Carlo techniques. For new document titles the prediction accuracy with eye movements, collaborative filtering, and their combination was significantly better than by chance. The best prediction accuracy still leaves room for improvement but shows that proactive information retrieval and combination of many sources of relevance feedback is feasible.
Combining eye movements and collaborative filtering for proactive information retrieval.
We study a new task, proactive information retrieval by combining implicit relevance feedback and collaborative filtering. We have constructed a controlled experimental setting, a prototype application, in which the users try to find interesting scientific articles by browsing their titles. Implicit feedback is inferred from eye movement signals, with discriminative hidden Markov models estimated from existing data in which explicit relevance feedback is available. Collaborative filtering is carried out using the User Rating Profile model, a state-of-the-art probabilistic latent variable model, computed using Markov Chain Monte Carlo techniques. For new document titles the prediction accuracy with eye movements, collaborative filtering, and their combination was significantly better than by chance. The best prediction accuracy still leaves room for improvement but shows that proactive information retrieval and combination of many sources of relevance feedback is feasible.
Two%2DWay Latent Grouping Model for User Preference Prediction.
We introduce a novel latent grouping model  for predicting the relevance of a new document  to a user. The model assumes a latent  group structure for both users and documents.
Associative Clustering for Exploring Dependencies between Functional Genomics Data Sets.
Part 2, vol. 2, no. 3, pages 203–216, 2005. This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of Helsinki University of Technology’s products or services. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by writing to pubs-permissions@ieee.org. By choosing to view this document, you agree to all provisions of the copyright laws protecting it. Thank you. c ○ 2005 IEEE. Reprinted with permission.
Self%2Dorganizing map%2Dbased discovery and visualization of human endogenous retroviral sequence groups.
map-based discovery and visualization of human endogenous
Exploratory modeling of yeast stress response and its regulation with gcca and associative clustering.

Discriminative clustering.
A distributional clustering model for continuous data is reviewed and new methods for optimizing and regularizing it are introduced and compared. Based on samples of discrete-valued auxiliary data associated to samples of the continuous primary data, the continuous data space is partitioned into Voronoi regions that are maximally homogeneous in terms of the discrete data. Then only variation in the primary data associated to variation in the discrete data a#ects the clustering; the discrete data &#034;supervises&#034; the clustering. Because the whole continuous space is partitioned, new samples can be easily clustered by the continuous part of the data alone. In experiments, the approach is shown to produce more homogeneous clusters than alternative methods. Two regularization methods are demonstrated to further improve the results: an entropy-type penalty for unequal cluster sizes, and the inclusion of a model for the marginal density of the primary data. The latter is also interpretable as special kind of joint distribution modeling with tunable emphasis for Preprint submitted to Neurocomputing 23 November 2004 discrimination and the marginal density.
Sequential information bottleneck for finite data.

Associative Clustering.
This report contains derivations which did not fit into the paper [3]. Associative clustering (AC) is a method for separately clustering two data sets when one-to-one associations between the sets, implying statistical dependency, are available. AC finds Voronoi partitionings that maximize the visibility of the dependency on the cluster level. The main content of this paper are technical results related to the algorithm: A Bayes factor interpretation of AC, derivation of gradients for optimizing AC with a smoothing trick, and the connection of AC objective to mutual information. I.
Sequential information bottleneck for finite data.



Mining massive document collections by the WEBSOM method.

Improved learning of Riemannian metrics for exploratory analysis.
We have earlier introduced a principle for learning metrics, which shows how metricbased methods can be made to focus on discriminative properties of data. The main applications are in supervising unsupervised learning to model interesting variation in data, instead of modeling all variation as plain unsupervised learning does. The metrics are derived by approximations to an information-geometric formulation. In this paper we review the theory, introduce better approximations to the distances, and show how to apply them in two di#erent kinds of unsupervised methods: prototype-based and pairwise-distance based. The two examples are self-organizing maps and multidimensional scaling (Sammon&#039;s mapping). Key words: Information geometry, information visualization, learning metrics, multidimensional scaling, self-organizing map 1 
Principle of Learning Metrics for Exploratory Data Analysis.


Visualizations for Assessing Convergence and Mixing of MCMC.
Abstract. Bayesian inference often requires approximating the posterior distribution with Markov Chain Monte Carlo (MCMC) sampling. A central problem with MCMC is how to detect whether the simulation has converged. The samples come from the true posterior distribution only after convergence. A common solution is to start several simulations from different starting points, and measure overlap of the different chains. We point out that Linear Discriminant Analysis (LDA) minimizes the overlap measured by the usual multivariate overlap measure. Hence, LDA is a justified method for visualizing convergence. However, LDA makes restrictive assumptions about the distributions of the chains and their relationships. These restrictions can be relaxed by a recently introduced extension. 1

Trustworthiness and metrics in visualizing similarity of gene expression.
Research article Trustworthiness and metrics in visualizing similarity of gene expression
Discriminative Clustering Optimal Contingency Tables by Learning Metrics.
The learning metrics principle describes a way to derive metrics  to the data space from paired data. Variation of the primary data  is assumed relevant only to the extent it causes changes in the auxiliary  data. Discriminative clustering finds clusters of primary data that are  homogeneous in the auxiliary data. In this paper, discriminative clustering  using a mutual information criterion is shown to be asymptotically  equivalent to vector quantization in learning metrics. We also present  a new, finite-data variant of discriminative clustering and show that it  builds contingency tables that detect optimally statistical dependency  between the clusters and the auxiliary data. A finite-data algorithm is  demonstrated to outperform the older mutual information maximizing  variant.
Learning More Accurate Metrics for Self%2DOrganizing Maps.
Abstract. Improved methods are presented for learning metrics that measure only important distances. It is assumed that changes in primary data are relevant only to the extent that they cause changes in auxiliary data, available paired with the primary data. The metrics are here derived from estimators of the conditional density of the auxiliary data. More accurate estimators are compared, and a more accurate approximation to the distances is introduced. The new methods improved the quality of Self-Organizing Maps (SOMs) significantly for four of the five studied data sets. 1

Clustering Based on Conditional Distributions in an Auxiliary Space.
We study the problem of learning groups or categories that are local

Electronic editor automatic content%2Dbased sequential compilation of newspaper articles.

Data Visualization and Analysis with Self%2DOrganizing Maps in Learning Metrics.

Neighborhood Preservation in Nonlinear Projection Methods An Experimental Study.
. Several measures have been proposed for comparing nonlinear  projection methods but so far no comparisons have taken into  account one of their most important properties, the trustworthiness of  the resulting neighborhood or proximity relationships. One of the main  uses of nonlinear mapping methods is to visualize multivariate data, and  in such visualizations it is crucial that the visualized proximities can be  trusted upon: If two data samples are close to each other on the display  they should be close-by in the original space as well. A local measure  of trustworthiness is proposed and it is shown for three data sets that  neighborhood relationships visualized by the Self-Organizing Map and  its variant, the Generative Topographic Mapping, are more trustworthy  than visualizations produced by traditional multidimensional scalingbased  nonlinear projection methods.  1 
Clustering Gene Expression Data by Mutual Information with Gene Function.
. We introduce a simple on-line algorithm for clustering paired
Clustering by Similarity in an Auxiliary Space.
We present a clustering method for continuous data. It defines local clusters into the (primary) data space but derives its similarity measure from the posterior distributions of additional discrete data that occur as pairs with the primary data. As a case study, enterprises are clustered by deriving the similarity measure from bankruptcy sensitivity. In another case study, a content-based clustering for text documents is found by measuring differences between their metadata (keyword distributions) . We show that minimizing our Kullback-Leibler divergencebased distortion measure within the categories is equivalent to maximizing the mutual information between the categories and the distributions in the auxiliary space. A simple on-line algorithm for minimizing the distortion is introduced for Gaussian basis functions and their analogs on a hypersphere.
Metrics that Learn Relevance.
We introduce an algorithm for learning a local metric to a continuous input space that measures  distances in terms of relevance to the processing task. The relevance is defined as local changes in  discrete auxiliary information, which may be for example the class of the data items, an index of  performance, or a contextual input. A set of neurons first learns representations that maximize the  mutual information between their outputs and the random variable representing the auxiliary information.  The implicit knowledge gained about relevance is then transformed into a new metric of the  input space that measures the change in the auxiliary information in the sense of local approximations  to the Kullback-Leibler divergence. The new metric can be used in further processing by other  algorithms. It is especially useful in data analysis applications since the distances can be interpreted  in terms of the local relevance of the original variables.  1 Introduction  The success of unsuper...
Websom for Textual Data Mining.
New methods that axe user-friendly and efficient axe needed for guidance among the masses of textual information available in the Internet and the World Wide Web. We have developed a method and a tool called the WEBSON which utilizes the self-organizing map algorithm (SON) for organizing laxge collections of text documents onto visual document maps. The approach to processing text is statistically oriented, computationally feasible, and scalablever a million text documents have been ordered on a single map. In the axticle we consider different kinds of information needs and tasks regaxding organizing, visualizing, seaxching, categorizing and filtering textual data. Furthermore, we discuss and illustrate with examples how document maps can aid in these situations. An example is presented where a document map is utilized as a tool for visualizing and filtering a stream of incoming electronic mail messages.
Methods for interpreting a self%2Dorganized map in data analysis.
. The Self-Organizing Map (SOM) can be used for forming overviews of multivariate data sets and for visualizing them on graphical map displays. Each map location represents certain kinds of data items and the value of a variable in the representations can be visualized in the corresponding locations on the map display. Such component plane displays contain all the information needed for interpreting the map but information about the relations of the variables remains implicit. We have developed methods that visualize explicitly the contribution of each variable in the organization of the map at different locations. It is also possible to measure the contribution of each variable in the cluster structure within an area of the map to summarize, for instance, the characteristics of clusters. 1. Introduction  The SOM algorithm [2, 3] forms a mapping of a usually two-dimensional map lattice into the high-dimensional data space. There is a model vector connected to each point of the discrete...
WEBSOM %2D Self%2Dorganizing maps of document collections.
Computational analysis of natural language is often focused on the syntactic structure of language– often with no regard to the overall context and area of expertise of the analyzed text. In this paper, we present a means of analyzing text documents from various areas of expertise to discover groups of thematically similar texts with no prior information about the topics. The presented results show how a relatively simple keyword analysis combined with a SOM projection can be very descriptive in terms of analyzing the contextual relationships between documents and their authors. 1
Self%2Dorganized Formation of Various Invariant%2Dfeature Filters in the Adaptive%2Dsubspace SOM.
The Adaptive-Subspace SOM (ASSOM) is a modular neural-network architecture, the modules of which learn to identify input patterns subject to some simple transformations. The learning process is unsupervised, competitive, and related to that of the traditional SOM (Self-Organizing Map). Each neural module becomes adaptively specific to some restricted class of transformations, and modules close to each other in the network become tuned to similar features in an orderly fashion. If different transformations exist in the input signals, different subsets of ASSOM units become tuned to these transformation classes.
Computationally Efficient Approximation of a Probabilistic Model for Document Representation in the WEBSOM Full%2DText Analysis Method.

Very Large Two%2DLevel SOM for the Browsing of Newsgroups.
. On January 19, 1996 we published in the Internet a demo of how to use Self-Organizing Maps (SOMs) for the organization of large collections of full-text files. Later we added other newsgroups to the demo. It can be found at the address http://websom.hut.fi/websom/. In the present paper we describe the main features of this system, called the WEBSOM, as well as some newer developments of it. 1 Introduction  When organizing large collections of free-form full-text document files that contain no keywords, e.g. the newsgroups in the Internet, it is difficult to base their analysis on traditional search expressions. The main information one can resort to in the classification of such documents is statistical. SOMs of document collections have previously been constructed on the basis of their word histograms (published works are [5], [6], [7], [10], [11], [12]). Thereby, however, the size of the selected vocabulary cannot be large. In other studies (cf., e.g., [1], [2], [8], [9], [11], and...
Comparing Self%2DOrganizing Maps.
. In exploratory analysis of high-dimensional data the selforganizing map can be used to illustrate relations between the data items. We have developed two measures for comparing how different maps represent these relations. The other combines an index of discontinuities in the mapping from the input data set to the map grid with an index of the accuracy with which the map represents the data set. This measure can be used for determining the goodness of single maps. The other measure has been used to directly compare how similarly two maps represent relations between data items. Such a measure of the dissimilarity of maps is useful, e.g., for analyzing the sensitivity of maps to variations in their inputs or in the learning process. Also the similarity of two data sets can be compared indirectly by comparing the maps that represent them. 1 Introduction  The self-organizing map (SOM) [4, 5] algorithm forms a kind of a nonlinear regression of an ordered set of reference vectors m i , i =...
Self%2DOrganizing Maps of Document Collections A New Approach to Interactive Exploration.
Powerful methods for interactive exploration and search from collections of free-form textual documents are needed to manage the ever-increasing flood of digital information. In this article we present a method, WEBSOM, for automatic organization of full-text document collections using the self-organizing map (SOM) algorithm. The document collection is ordered onto a map in an unsupervised manner utilizing statistical information of short word contexts. The resulting ordered map where similar documents lie near each other thus presents a general view of the document space. With the aid of a suitable (WWWbased) interface, documents in interesting areas of the map can be browsed. The browsing can also be interactively extended to related topics, which appear in nearby areas on the map. Along with the method we present a case study of its use. Keywords: data visualization, document organization, full-text analysis, interactive exploration, self-organizing map. Introduction Finding relev...
Winner%2Dtake%2Dall networks for physiological models of competitive learning.

